Property-Based Matrix Testing in Java

I've been using the same technique to accomplish property-based matrix testing in Java for a long time.
It's not very pretty, or clean, but it gets the job done. 😎

Using JUnit's ParameterizedTest and a MethodSource annotations to point to a method as an argument supplier that spits out a stream representing my matrix.
It's pretty straightforward, but the more parameter types I have in my matrix, the harder it is to read or write the method supplying them. 😵

Let's take a look.
First, these are the two types for our matrix, these should spit out 6 test cases:

  enum Direction {
    INCOMING,
    OUTGOING
  }

  enum Status {
    SUCCESS,
    FAILURE,
    WAITING
  }

Implementing a matrix from these Enums with JUnit's ParameterizedTest and a MethodSource:

  @ParameterizedTest
  @MethodSource("getArguments")
  void using_junit_parameterized_test_with_method_source(
      final Direction direction, final Status status) {
    assertTrue(true);
  }

  static Stream<Arguments> getArguments() {
    return Stream.of(Direction.values())
        .flatMap(d -> Stream.of(Status.values()).map(s -> arguments(d, s)));
  }

As you can see, adding members to the existing Enums will dynamically increase the matrix,
and therefore the number of tests performed, there's no need to modify the test code.

But, adding a third type to the matrix, and the getArguments method, will start losing its readability.

Lately, I discovered JUnit Pioneer, which is a JUnit 5 Extension Pack.
Using its CartesianProductTest and CartesianEnumSource annotations we can implement the same exact matrix simply and elegantly: 😃

  @CartesianProductTest
  @CartesianEnumSource(Direction.class)
  @CartesianEnumSource(Status.class)
  void using_junit_pioneer_cartesian_product_test_with_enum_source(
      final Direction direction, final Status status) {
    assertTrue(true);
  }

This will spit out the same matrix, only now, adding a third element is quite simple, just add another CartesianEnumSource annotation.
You can find other types of sources beside Enums, in JUnit Pioneer's Documentation.

As demonstrated in this repository, executing both matrix tests will print out:

[INFO] '-- JUnit Jupiter [OK]
[INFO]   '-- Property Based Matrix Test [OK]
[INFO]     +-- using junit pioneer cartesian product test with enum source (Direction, Status) [OK]
[INFO]     | +-- [1] INCOMING, SUCCESS [OK]
[INFO]     | +-- [2] INCOMING, FAILURE [OK]
[INFO]     | +-- [3] INCOMING, WAITING [OK]
[INFO]     | +-- [4] OUTGOING, SUCCESS [OK]
[INFO]     | +-- [5] OUTGOING, FAILURE [OK]
[INFO]     | '-- [6] OUTGOING, WAITING [OK]
[INFO]     '-- using junit parameterized test with method source (Direction, Status) [OK]
[INFO]       +-- [1] INCOMING, SUCCESS [OK]
[INFO]       +-- [2] INCOMING, FAILURE [OK]
[INFO]       +-- [3] INCOMING, WAITING [OK]
[INFO]       +-- [4] OUTGOING, SUCCESS [OK]
[INFO]       +-- [5] OUTGOING, FAILURE [OK]
[INFO]       '-- [6] OUTGOING, WAITING [OK]

You can check out the code for this tutorial in Github.

👋 See you in the next tutorial 👋